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THIS STUDY EXAMINES A PROCEDURE FOR USING THE WRITTEN 1 

CASE RECORD AS A CRITERION FOR EVALUATING REHABILI TATION I 

COUNSELOR PERFORMANCE IN STATE "DVR" (STATE-FEDERAL GENERAL j 

VOCATIONAL REHABILI TATION PROGRAM ) AGENCIES . THE RESEARCHERS 

OUTLINE CERTAIN OBJECTIVES (TECHNICAL, AGENCY SERVICES, AND J 

STAFF DEVELOPMENT) AS DESIRABLE IN THE EVALUATION PROCEDURES j 

OF SUCH AGENCIES. BASED UPON EARLIER WORK DESCRIBING CRITICAL S 

REQUIREMENTS IN REHABILITATION, THE STRUCTURED CASE REVIEW j 

BLANK (SCRB) WAS DEVELOPED. SUPERVISORS WERE TRAINED TO USE 
THIS BLANK WHICH ASSESSED NINE AREAS OF COUNSELOR PERFORMANCE 
AND FOUR ADDITIONAL ASPECTS OF CLERICAL COMPETENCE. RATINGS j 

OBTAINED FROM 26 SUPERVISORS OF 143 COUNSELORS WERE f 

INTERCORRELATED WITH EACH OTHER AND WITH OTHER CRITERIA . j 

RATINGS OF CASE RECORDS ARE ABOUT AS RELIABLE AS OTHER TYFES ) 

OF RATINGS USED TO EVALUATE COUNSELOR PERFORMANCE. BECAUSE AN j 

ANALYSIS OF RELATIONSHIPS WITHIN THE SCRB HAS SHOWN RATHER j 

HIGH INTERCORRELATIONS AMONG THE DIMENSIONS, GROUPS, AND 1 

TOTAL SCORES OF THE BLANK, SCRB RATINGS HAVE BEEN REJECTED IN j 

THE DIFFERENTIAL ANALYSIS OF COUNSELOR PERFORMANCE. THIS ] 

DOCUMENT WAS PUBLISHED IN "THE CRITERIA PROBLEM IN 

REHABILITATION COUNSELING " AS CHAPTER III (PP. 16-27), j 

APPENDIX B (PP. 71-72), APPENDIX C (PP. 73-82), APPENDIX H J 

(PP. 120-122), AND REFERENCES (PP. 123-129) . (PS) I 
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CHAPTER III 

STRUCTURED CASE REVIEW STUDY 



The Nature of the Problem 

An Important part of our total study was the examination of a procedure 
for using written case records as a criterion for evaluating rehabilitation 
counselor performance in state DVR agencies. In planning this study, we as* 
sumed that certain objectives were desirable in the evaluation procedures of 
such agencies. Consequently, it may be helpful, in order to better understand 
why this study was undertaken, as well as the resulting information, to begin 
by considering what these objectives were. 

1. Agency Services Objective ; Evaluation of counselor performance 
should ultimately be based on how well the services he provides 
achieve agency goals as reflected in agency policy and philosophy. 
Such goals are the agency’s reason for existence. 

2. Staff Development Objective : Evaluation should permit differen- 

tial analysis of counselor performance to aid him in overcoming 
areas of weak performance. This would also permit in-service 
training, state conferences, etc., to bear on any supplemental 
training the staff as a whole might need. 



3. Technical Objective : Evaluation procedures should be as reliable 

and economical as is consistent with the above objectives. 

Since the rehabilitation counselor's job is highly complex, descriptive 
research must first be done to define criteria of performance that can achieve 
the stated objectives of performance evaluation and are feasible for use in 
later research studies in such areas as selection, training, and program plan- 
ning. In this study, the Structured Case Review Blank (SCRB) (Appendix B), 
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ratings of counselor case records on selected variables, and their relationships 
with other immediate and intermediate-level criteria was of central interest. 

The SCRB rating form was developed and studied because (1) the previous survey 
of Muthard and Miller (1963) revealed that DVR agencies already use case review 
to evaluate certain dimensions of counselor performance; however, there had 
been no research on this criterion; (2) we thought the SCRB might permit dif- 
ferential analysis of counselor performance, in keeping with the staff develop- 
ment objective of such evaluations; and (3) the experiences and findings from 
rating case material with the SCRB might indicate the feasibility of evaluating 
counselor performance in this manner. 



Procedures of the Study 

Selecting Dimensions and Format for the SCRB : The variables that were 

judged in the record were selected from an earlier Iowa study of critical coun- 
seling behavior. Jaques' (1959) study established the importance of these di- 
mensions in considering the work of the rehabilitation counselor. They were 
placed together in three logical groups in the case review blank: Group I: 

Creation of a therapeutic climate - interacting; Group II: Information giving, 

gathering - evaluating; Group III: Structuring. Each grouping has three case 

service variables. 



The nine case service variables required paired judgments of (1) need and 
(2) amount provided in particular cases. These were judged on a seven-step 
scale: 

1: 2 : 3 : 4 : 5 : 6 : 7 

low average high 

The groups and items are listed below: 



Group I 

1. Need to create a therapeutic climate. 

2. Need to work with other agencies or professional associates. 

3. Neted to create a learning situation for client or significant others. 

Group II 

4. Need for information from others as well as client. 

5. Need for giving information to client about vocations or rehabilitation 
facilities. 

6. Need for a careful, extended evaluation of information. 

Group III 

7. Need to arrange for this client. 

8. Need for time in this case. 

9. Need to orient client to services. 



An individual definition of each variable was presented to supervisors, 
both in small group discussions and a training manual. We also suggested t;he 
kinds of case record data upon which the amount of need and the amount of case 
service provided could be rated for each dimension. 
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It was assumed that by taking the discrepancy between scores for the amount 
of need and the amount of provision, a "performance index" could be derived. In 
a minus difference, need was judged greater than amount provided, indicating a 
deficient provision. In a plus difference, amount provided was judged greater 
than need, indicating an over-concern. Thus, scores on each variable reflect 
the degree to which the counselor has been deficient or over -concerned, with no 
discrepancy (zero difference between (1) need and (2) counselor provision rat- 
ings) indicating "perfect" performance. Performance can suffer not only in be- 
ing "deficient" but also in being "over -concerned" with a service for which there 
is no great need. 

On the front of the SCRB, we listed four aspects on which to rate the cleri- 
cal handling of a case. Each was rated on a one- to seven-step scale. These 
were added to secure total case points. They included: 

1. Clerical Detail : Whether a case was rated low or high (with four as 

the average) depended upon the inclusion and degree of completeness 
of agency forms appropriate to a case in this status. That is, medi- 
cal and special examinations, psychological appraisal, social and vo- 
cational histories, financial forms, et^. , had been completed with 
care. J - 

\ 

2. Organization of Case : In rating this, the supervisor focused upon 

the degree to which the case was organized in accordance with agency 
directives. For example, notes, medical reports and other data were 
arranged to facilitate handling by any individuals who must work with 
the case. This was for cases as a whole. 



! 



3. Continuity of Notes : This concerned the amount of integration and 

continuity on case notes. The supervisors considered whether notes 
for different interviews and the various activities of the rehabili- 
tation process were difficult to follow. They also evaluated how 
adequately the counselor reported the rationale for his actions and 
conclusions. 



4. Currency of Notes : Since cases in various statuses (e.g. doing a 

medical work-up or in training) might not require current notes, the 
currency of notes was judged according to case status, to some extent. 

Selecting Supervisors and Counselors : A major consideration for the design 

was that the study should permit reasonable generalization to the population of 
DVR agencies. Since the states are autonomous with respect to rating counselor 
performance, the results from one state would be rather highly restricted to that 
state only. Relationships between SCRB scores and other criteria could be ex- 
pected to fluctuate from state to state. This would be due partly to smaller 
numbers involved in the analysis and partly to individual state practices. In 
line with this consideration, the six states used in the total criteria study 
were considered as one sample, representing the population of such agencies. 

To have be in included in the overall study, a counselor must have: (1) worked 
at least one year for the agency; (2) worked in an office where at least two 
other counselors could give co-worker ratings; (3) been in the vocational re- 
habilitation program; and (4) not worked in a highly specialized setting with 
unique work tasks (e.g. mental hospital or school for the deaf). 
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The number of participating counselors who met these criteria was 143 with 

,l!T^ iS ° r8 J Sin f® the assignment of ratings on the SCRB was time -consuming 
(about five and one-half hours per counselor), it was impractical to have all * 

£™ COim i rated on this blank. A further sample of 56 counselors was, there 

fore, selected by using the Co-Worker Rating Blank (Appendix C). Those in the 
top ®r bottom 25 per cent of the counselors in each state met the first criteria 
n addition, counselors were selected so that a supervisor rated no more than 

JOZ^a' e * However ’ In state which had only two supervisors, each supervisor 
rated four counselors). Since several supervisors failed to provide ratings on 
clerical completeness, there were only 41 counselors rated in this area. The 
ratings by co-workers and supervisors, as well as Job Satisfaction Inventories 
(Appendix D), were usually obtained by mail, with individual "kits" made up and 
coded for each counselor or supervisor. 

gater Training and Case Selection: The SCRB reliability data and the train- 

a? 4 ° ®“ pervis ® rs was doiie in small groups. After supervisors had finished 

Review ^ir Ple reli f bi ^y cases > they were given a supply of Structured Case 
w I Blanks, a manual of instruction, and asked to rate assigned counselors. 
Making the ratings later in their district offices, the supervisors selected nine 
cases from each assigned counselor's caseload, as follows. At random, three 

sel *f e * fro ” Status 1 cases, three cases were selected frim the Ser- 
vice Statuses (2 through 6), and three from Pre -employment Status (usually Status 

u i 1 8 f nple °5 nin * re P resents ab <>ut a ten per cent sample, generally, 

® ounse } or s caseload in these statuses. The blanks were mailed directly 
to the investigator without computation of scores by the supervisors. 

S coring aqd Analysing Ratings : A counselor's case review score, for any 

dimension of the blank, was taken as the absolute sum of discrepancy scores * 
across all nine cases. Scores on all variables, except two, were converted to 
standard scores within each state. In effect, this eliminated inter-state dif- 
ferences and made all converted counselor scores on any variable comparable both 
between and within states. Ratings given by state administrators had already 
been changed to percentile ranks of counselors within states and were left as 
percentile ranks. Structured Case Review Blank scores, available on 56 counse- 
lors, were left as given by supervisors, since the relationship of SCRB scores 
ovgr states, in order to increase the generalities of the results, was of 
prime interest . 

The SCRB scores and all other variables were plotted to inspect for line- 
arity. Meeting the linear criterion, the case review scores were then corre- 
lated with all other variables, using the Pearson Product -Moment correlation. 

These correlations were for the 56 counselors who had been rated on the SCRB* 
however, such correlations had to be corrected since the counselors included’ 
had been selected according to Co-Worker Ratings. This was accomplished through 
a procedure outlined by Gulliksen (1950) and Thorndike (1949). Essentially, the 
task involved correcting the correlation between two variables for selection 
made on a third. Such a procedure yields an estimate of the correlation that 
would have occurred had all 143 counselors been rated. 
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Results 

In general, this study attempted to answer the question, "Can a case rec- 
ord evaluation form be developed to provide a criterion for evaluating rehabi- 
litation counselor performance in state agencies?" hore specifically, it at- 
tempted to provide answers to the following questions: (1) How reliable and 

interrelated are dimension, group, and total scores of the SCRB criterion in 
evaluating rehabilitation counselor performance in state agencies? What im- 
plications can be drawn from these findings? (2) Considering dimension, 
group, and total scores, what is the relationship of such a criterion to other 
criteria of performance? What implications can be drawn from these findings? 

Wine other criteria were examined in relationship to this criterion. They 
were: (1) co-worker ratings; (2) supervisor ratings; (3) present state rat- 

ings; (4) job satisfaction; (5) size of caseload; (6) caseload velocity; 

(7) number of 12 closures; (8) number of 13 closures; and (9) number of 15 
closures. These variables are described in Chapter IV. 



TABLE 5 

ESTIMATES OF RELIABILITY FOR TOTAL AND SUBSCORES 
OF STRUCTURED CASE REVIEW BLANK ^Sb) 1 



Description of Case Review Rating 



Between Raters Between Raters 
Included (11 Extracted (2) 



(1) Creation of a therapeutic climate. 

(2) Working with other agencies or pro- 
fessional associates. 

(3) Creating a learning situation for 
client or significant others. 

(4) Getting information from others as 
well as client. 

(5) Giving information to client about 
vocations and/or rehabilitation fa- 
cilities. 

(6) Doing a careful, extended evaluation. 

(7) Arranging for the client. 

(8) Giving sufficient time to a case. 

(9) Orienting client to services. 

Group I (Sum of 1, 2, 3) 

Group II (Sum of 4, 5, 6) 

Group III (Sum of 7, 8, 9) 

TOTAL Score 



37* 


49 


49 


62 


00 


39 


36 


64 


30 


50 


55 


71 


58 


72 


70 


76 


49 


71 


57 


74 


53 


71 


59 


74 


64 


82 



Correlations have been corrected for selection on Co-Worker Ratings using a 
procedure recommended by Davis (Thorndike, 1949). 

£ 

Decimals have been omitted from correlations. 
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Reliability of the SCRB : Using intra-case variance as an error term. 

Table 5 presents the estimated reliability coefficients for dimension, group, 

I and total scores. Actually, two error variances were identified for each case j 

; review dimension. Since, normally, only one supervisor rates a counselor, the 

error which includes between-rater (within-case) variance, since it includes 
consistent rater differences, might be considered more appropriate than the in- 
teraction variance. However, it could also be argued that if supervisors were 
given longer training in using the blank (in this study, they were given about 
I one hour of training), consistent rater differences would tend to grow smaller. 

In this case, the error would become equivalent to interaction variance when 
such differences vanished. In any actual situation, therefore, it could be an- 
ticipated that the reliability coefficient would fall in the range between col- 
l umns (1) and (2) of Table 5 for a given dimension score on the SCRB. This would 
depend partly on the amount of training and familiarity a supervisor had with 
the blank, as well as the care he exercised in completing it. Five actual cases, 
representing different ages, sex, and disability, wero selected from the files 
j of a DVR agency. With identifying information disguised, these cases were rated 
l by all supervisors in the study; and reliability estimates were derived from 
[ the ratings of these "dummy" cases. 

I 

Table 5 reflects relatively less reliability on dimensions (1), (3), (4) 
t and (5) . The fact that ratings on dimensions (1) and (3) were made on material 

[ more difficult to define and more subjective in character probably accounts for 

I their relatively low reliability. Ratings on dimensions (4) and (5), however, 
i were on dimensions similar to what state supervisors often consider in their 
current unstructured approach to case review. Such coefficients, however, re- 
flect relatively little reliability among supervisor ratings on what constitutes 
\ "good" counselor performance for these dimensions. 

I Group and total scores were relatively more reliable since they are based 

i on scores over several dimensions. In addition, ratings on dimensions which 

were relatively more concrete and specific, dimensions (7) and (8), also tended 
to be more reliable. Qf all dimension ratings, those for dimension (8), "Giving 
1 sufficient time to a case," were the most reliable in this study. 

l ■ 

Relationships Within the SORB : In general, research on rating forms has 

revealed such constant errors as halo effect, error of central tendency, logical 
error, contrast error, and proximity error (Guilford, 1954). The relationships 
which exist among the dimensions of the SCRB provide information on how such 
errors may be operating in this procedure. 

Table 6 provides the intercorrelations, within the SCRB, of dimension j 

scores and the total score. It is apparent from this table that scores on dif- 
ferent dimensions within the blank have moderate to high intercorrelations and 
| probably reflect a considerable amount of constant error • 

[ ' 

| The range of correlations in Table 6 for individual dimension scores ex- 

I tends from .63 to .83, with a median correlation of .73. When the intercorrela- 
tions over the five dummy cases used in the reliability study were computed, the 
range extended from .14 to .85, with a median of .55. The correlations of Table 
6 were obtained when the supervisors knew whom they were rating and, consequently, 

I reflect halo effect to some extent. It is reasonable to suppose, however, that 
halo operated to a lesser extent for the dummy cases and thus correlations were 
reduced in some instances. 



TABLE 6 
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INTERCORRELATIONS AMONG CASE REVIEW DIMENSIONS SCORES 
AS WELL AS TOTAL SCORE (N=56) 



Description of Case Review Ratings 

(1) Creation of a therapeutic climate. 



Case Review Ratings 



( 2 ) Working with other agencies or pro- 
fessional associates - 



(3) Creating a learning situation for 
client or significant others. 



(4) Getting information from others as 
well as client. 



(5) Giving information to client about 
vocations and/or rehabilitation fa* 
cilities. 



(t) Doing a careful, extended evaluation. 

(7) Arranging for the client. 

(8) Giving sufficient time to a case. 

(9) Orienting the client to services. 

(10) TOTAL Score 



(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


(9) 


(10] 


77* 


74* 


67* 


79 


83* 


59 


61 


83 


74 


80* 


62 


76 


79 


64 


57 


83 




74* 


76* 


82 


76 


67* 


•v* 

/ 4 


87 






66* 


76* 


72* 


61* 


56 


79 








75* 


71* 


63* 


78* 


81 










77* 


75* 


65 


88 












61 


68 


85 



56 73 

75 



Decimals have been omitted from correlations. 

^Correlations corrected for inclusion of variable. 

*These correlations reflect relatively more halo effect. Significance of dif- 
ferences from correlations for "dummy" cases was established by (-1.64) (S.D. 
of Zr for N-56), in which correlations of dummy cases were treated as hypothe- 
sized values. Differences were significant at .05 level (Blalock, 1960). 
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Assuming that significant differences between "dummy" and "actual" inter- 
correlations reflect this halo effect, Table 6 indicates those dimensions which 
appeared most susceptible to this error. From Table 6, we note that when super- 
visors rate counselors they actually know - as compared to dummy cases - ratings 
on dimensions (4) and (5) have significantly higher correlations with most of 
the other ratings. Apparently, supervisor impressions of how well the counselor 
collects and gives information form a basis for halo effect to operate on other 
ratings of his written case records. Thirteen of the 19 significantly higher 
correlations were found on these two dimensions. It is also interesting to note 
that ratings on uhe counselor's success in "creating a therapeutic climate" had 
higher correlations with ratings on how well he worked with other agencies and 
associates, as well as arranged things for the client. Of course, halo proba- 
bly operated within all dimensions to some extent, but dimensions (4) and (5) 
appear to be the nucleus of such halo. 

We can also assume that logical error, i.e«, the raters' tendency to give 
similar ratings for traits that seem to them to be logically related, accounted 
for part of the high inter cor relations within the blank.- For example, dimen- 
sion (2) ("Working with other agencies or professional associates") correlates 
quite highly (.80) with dimension (4) ("Arranging for the client") (.79). 

These dimensions may well have a strong logical relationship in the supervisor's 
mind, and thus the ratings tend to be more similar. 

Because of the high inhere orrelat ions within the blank, dimension ratings 
do not cluster into the groups which Jaques (1959) and we used. Tables 7 and 8 
present the intercorrelations of dimension with group scores and within the 
group scores themselves. The system of grouping used here clearly did not meet 
the criterion of independence between groups. That is, ratings within a group 
do not appear to correlate higher with each other than with other ratings or 
groups. It may be that constant errors, particularly halo and logical errors, 
are operating to obscure such groupings, but such errors are a part of any actual 
rating scheme and so must be taken into account. Because of the rather uniformly 
high intercorrelations within the SCRB, neither cluster nor factor analysis wa°s 
feasible. 

The supervisors in this study also made ratings on clerical completeness 
- case points - for the case records selected on each counselor. (As pointed 
out earlier, some supervisors did not rate case points for several counselors, 
and consequently the number of counselors was reduced to 41). As Table 9 shows, 
case points or ratings on clerical completeness have moderate relationships, on 
the average, with ratings on other dimensions. Dimensions (1), (3) and (9) cor- 
relate significantly higher with clerical completeness than do the other dimen- 
sions. Since dimensions (1), (3) and (9) were more difficult to define and rate, 
we might conjecture that supervisors confronted with ambiguous rating dimensions 
tended to rely on how completely the counselor filled out forms, made note en- 
tries, etc. 

Relationships Between SCRB Ratings and Nine Criteria : Up to now, the an- 
alysis has focused on the blank itself. In this section, however, ratings on the 
SCRB are inter correlated with nine other criteria of counselor performance. Exami- 
nation of Table J (Appendix H) reveals that overall case review ratings do not 
correlate with other criteria. In only one instance, that of "Orienting the 
client to services" and "Supervisor ratings," was a significant correlation found. 







TABLE 7 



CORRELATIONS OF CASE REVIEW DIMENSION SCORES 
WITH GROUP SCORES (N-56) 



Groups 



Description of Case Review Ratines 


I 

Sum of 
1. 2, 3 


II 

Sum of 
4, 5, 6 


III 
Sum < 
7, 8, 


(1) 


Creation of a therapeutic climate. 


it 

86 


81 


77 


(2) 


Working with other agencies or pro- 
fessional associates. 


83 


80 


76 


(3) 


Creating a learning situation for 
client or significant others. 


78 


86 


83 


(4) 


Getting information from others as 
well as client. 


82 


74 


72 


(5) 


Giving information to client about 
vocations and/or rehabilitation fa- 
cilities. 


73 


76 


82 


(6) 


Doing a careful, extended evaluation. 


85 


84 


84 


(7) 


Arranging for the client. 


86 


82 


73 


<8> 


Giving sufficient time to a case. 


68 


75 


64 


(9) 


Orienting the client to services. 


69 


75 


68 



Decimals have been omitted from correlations. 
^Correlations corrected for inclusion of variable. 
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TABLE 8 



INTERCOFJftELATIONS AMONG CASE REVIEW GROUP SCORES 
AS WELL AS TOTAL SCORE (N-56) 



i 

Description off Case Review Ratines 
| Group I 

I (1) Creation of a therapeutic climate. 

I (2) Working with other agencies or pro- 

I fessional associates. 

I 

| (3) Creating a learning situation for 

I client or significant others. 



| Group II 

| 

| (4) Getting information from others as 

I well as client. 

I 

| (3) Giving information to client about 

| vocations and/or rehabilitation fa- 

15 cities. 

| 

I (6) Doing a careful, extended evaluation. 

| 

& 

| 

| Group III 

| 

I (7) Arranging for the client. 

I (8) Giving sufficient time to a case. 

| (9) Orienting the client to services. 



\ ft 

Decimals have been omitted from correlations. 
| Corrected for inclusion of variable. 

[ 

i 

« 

t 

i 

i 



Group II Group III Total* 
89* 85 89 



89 93 



90 



i 




i 
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TABLE 9 

CASE REVIEW DIMENSION SCORES AND TOTAL SCORE 
WITH CASE POINTS (N-41) 



Description of Case Review Ratings 

(1) Creation Of a therapeutic climate. 

(2) Working with other agencies or pro- 
fessional associates. 

(3) Creating a learning situation for client 
or significant others. 

(4) Getting information from others as 
well as client. 

(5) Giving information to client about 
vocations and/or rehabilitation fa- 
cilities. 

(6) Doing a careful, extended evaluation. 

(7) Arranging for the client. 

(8) Giving sufficient time to a case. 

(9) Orienting the client to services 
TOTAL Score 



Case Points 



57** 


3 




(i 

\ ' \ 


33 


i 

• i 




Cl 


53 


i 

A 

h 

,i 


36 


! ^ 



39 

37 

43 

25 

54 * 

50 



^Decimals have been omitted from correlations. 

Correlations significantly higher (.05) than others in table. Significance 
established through a procedure developed by Hotelling (Blalock, 1960) 
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Since supervisors gave both ratings and SCRB scores, it is surprising that addi- 
tional SCRB ratings did not correlate significantly with the overall supervisor 
| ratings . One could anticipate that constant errors, as was the case within the 

blank itself, would tend to raise the correlations between case review scores 
and ratings given by the same supervisors. The unreliability in both criteria 
attenuated the correlations. In addition, the two ratings were many times given 
months apart and in different forms. Consequently, two different views of the 
counselor ware elicited: one a global rating based upon previous impressions, 

the other upon assessment of case records on structured dimensions. 

Group and total scores of the SCRB do not correlate to any extent with the 
! n ^ ne other criteria (Table K, Appendix H). The correlations between case points 
and scores on the nine criteria showed only one significant relationship. Pre- 
sent state ratings correlated .32 with case points (Table L, Appendix H). 



i 

Summary 

Based upon earlier work describing critical requirements in rehabilitation 
counseling, a case review blank, the SCRB, was developed. Supervisors were 
trained to use this blank which assessed nine areas of counselor performance 
and four additional aspects of clerical competence. Ratings obtained from 26 
I supervisors for 143 counselors were lntercorrelated with each other and nine 
other criteria. 

j 

Ratings of case records for evaluating counselor performance seem, in 
general, about as reliable as ratings used in other contexts (Guilford, 1954; 
j Taylor and Hastman, 1956); that is, the average inter-rater correlation was* 
between .50 to .60. Analysis of relationships within the SCRB has shown rather 
| high intercorrelations among the dimensions, groups, and total scores of the 
[ blank. Such high intercorrelations require the rejection of SCRB ratings for 
j differential analysis of counselor performance. Ratings on clerical complete- 
ness tend to have moderate correlations with other ratings on the SCRB. 

Scructured Case Review Blank ratings do not correlate with other criteria 
I examined in this study. 
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APPENDIX B 



STRUCTURED CASE-REVEEW BLANK 



identifying data: 

■ample no Cue no. 



Status 



Total D-Score 



duxu« 1^« Name 



Icricai Handling: 



o t j M • * \ ? S 



1. Clerical detail 

2. Organization of case 

3. Continuity of Notes 

4. Currency of Notes 



l i 2 « 3 « 4 I 5 « 6 » 7 
low average high 

i » 2 i 3 « 4.« 5_ « _6_ » _7_ 

low average high 

1 t 2 s 3 » 4 : 5_» 6j.7 

low average high 

1 » 2 « 3 « 4 « $.'±.'2. 
low average high 



>H*rapeutic Climate, Interaction 

ng. :$gp FOR CREATING A THERAPEUTIC CLIMATE ISt 
•atai 1) Reports showing personality problem 
2) Agency forms for overview 



$ v MOUNT OF THIS CASE-SERVICE PROVIDED: 
Data: 1) Explicit recognition in notes 
2) A plan to provide it 



i i 2 » 3 i 4 i 5 : 6:7 

low average high 



JL J-'J.' JL'2. 

low average high 



the NEED TO WORK WITH OTHER AGENCIES OR 

PROFESSIONAL ASSOCIATES IS 

Data: i) Requests for cooperation from others 

2) Needs of client your agency can't meet 

3) Whether or not client is involved with other agencies 



•JL «A‘ J-'JL'-L'-l'-Z. 

low average high 



i 



HE AMOUNT OF THIS CASE-SERVICE PROVIDED: . 
Data: 1) How well material needs are being met 
2) Notes and letters reflecting cooperation 



rvi 



& N EED TO CREATE A LEARNING SITUATION FOR CLIENT 

« SIGNIFICANT OTHERS IS ? • • - 

’ utai 1) Client's lack of prevccatlohai skills 
2) Lack of understanding by others 



AM OUNT OF THIS CASE-SERVICE PROVIDED . 

>«t * t) Recognition of necessary skills to be learned 

2) A plan to provide instruction and Information to others 



i.'i.'i.'i.'JL 1 6 : 7 _ 
low average high 



JL :_ 2 j , 3 _i JMJ.: _ 6 _:_Z. 

low average high 



-L'JL' JL'JL'J.' JL'-Z. 

low average high 



Group I D-Score 



Turn blank over to continue 
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Group II — Information Giving, Gathering, Evaluating 

4. a) THE NEED FOR INFORMATION FROM OTHERS AS WELL 

AS CLIENT IS! 

Data: 1) Has problems that cannot be solved 
without information 

2) Vagueness to case about problems and planning 



" L'\:. 



jh •> 



1 « 2 i _4. i J> i_6. i 7 
low average high 



b) THE AMOUNT OF THIS CASE-SERVICE PROVIDED! 
Data! 1) Actu al reports from others or client 
2) Planning to elicit Information 



5 . a) THE NEED FOR GIVING INFORMATION TO CLIENT ABOUT 

VOCATIONS AND/OR REHAB. FACILITIES ISi 

Data! 1) Significant changes are necessary 
2) Client knowledgeability 

b) THE AMOUNT OF THIS CASE-SERVICE PROVIDED! . . . . 
Data: 1) Tours, testing, Interviews, etc. 

undertaken to provide information 



JL »_2_« 3 i 4 i S i 6 t _7 

low average high 



-1 * JL J « JL *_! «_Z. 

low average high 



6. a) THE NEED FOR A CAREFUL, EXTENDED EVALUA TION ISi 
Data: 1) History of failure 

2) A problem with many implications 



b) THE AMOUNT OF THIS CASE-SERVICE PROVIDED! 
Data: 1) General manner In which counselor 
logically relates features of case 



■ 1 » 2 i _3 .i_4.i_ 5.! 6 t 7 
low average high 



_i * _2_ i _B_j _4_ i _5_ : j6_:_7_ 
low average high 



JL *-2.! X’-l* 

low average high 



Group II D-Score 



Group III — Structuring 

7 « a ) TEE NEED TO ARRANGE FOR THIS CLIENT IS: 

Datat 1) Level of Independent action 

2) Demonstrated ability to cope with similar situations 



b) THE AMOUNT OF THIS CASE-SERVICE PROVIDED: 
Datat 1) What arranging was done 

2) What planning has fallen through 
for lack of arranging 



1 i 2 i 3 i 4 i S ; 6i 7 
low average high 



_X «JL« 3 i 4 i S i { > 7 
krw average high 



8. a) TOE NEED FOR TIME IN THIS CASE IS:. i , » . a . a . e . a * 

n a * a . i\ D^i -i jL*_Z. 



Datat 1) Problems arising from lack of time 



low 



average 



high 



b) THE AMOUNT OF THIS CASE- SERVICE PROVIDED: lio.a.aea-r 

n r , — JL«-L 



Data: 1) Last Interview and frequency 



low 



average 



high 



9. a) THE NEED TO ORIENT THE CLIENT TO SERVICES ISt 
Datat 1) How well client should know goals 

2) How well client knows the role he plays 



b) THE AMOUNT OF THIS CASE-SERVICE PROVIDED: . . . 
Datat 1) Amount of orientation given 

2) Client actions that reflect good orientation 



JL , i. , J. , JL‘i.«i.t2 

low average high 



JL l i. l -3. l -ii _5 < 6 : 7 
low average high 



Group HI D- Score 



0 
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APPENDIX C 

CO-WORKER RATING BLANK 

i 

We are asking for your help in a research project which concerns the 
evaluation procedures used towards the work of the rehabilitation counselor. 
This project - under way since November 1959 and supported by VRA - we hope 
will help define some of the problems and pertinent dimensions by which such 
an evaluation can be approached. 

One variable we want to study in our project is the perceptions which 
counselors have about the work of fellow counselors. We feel that co-workers 
are perhaps in a good position to know some aspects of that work since they 
have a chance to interact and listen to each other - all of which helps them 
form some idea of how well the other counselor is performing his job. With 
this in mind, we thought that one important measure of counselor performance 
would be the perceptions of fellow workers. 



The blank should be completed in the following manner: 

1. On the last page of the blank, there is a list of names sectioned 
into district offices. In front of each name there is a number. 

2. From this list, select the counselors which appear in the same sec- 
tion as your name, and put their numbers in the squares at the top 
of page 2 - above the items to be rated. The initials, etc., of 
each person may be put below each number in order to help you remem- 
ber who they are. Please rate all counselors in that section, in- 
cluding yourself . (If you are a supervisor, please rate all the 
counselors in the section that has your district office in the title. 
You need not rate yourself.) 

3. On page 2, there are seven items of a counselor's work on which the 
people you selected should be compared. On the scale for each item, 
all the numbers you selected should be placed. 

B.g. : 12 : 2.4 : 9 : 8 : U_ : 7,3 

It most like I® llk « 

4. On each item, you will compare the behavior, as you see it, of each 
person you selected to the behavior of' the best counselors you have 
known - or to what you feel the behavior of the best counselor should 
be. In effect, you are comparing how "like" or similar to your ideal 
the behavior of the selected counselors corresponds. If it is *aaost 
like" your ideal, then the number should be placed near or at the 

like" end, depending on how nearly you feel it is like. If it 
is "least like," then the other end of the scale should be used, de- 
pending on how strongly you feel it is "least like" your ideal* 
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5. Work on one ltea at • tin.; put «U m-bor. .el«ct«l on on* »e«U 
and then «ove to the neat its*. 



Your reply will be held in atricteet confidence; only the investigator* 

have the coding key which identifies each blank. In ^ JJ**?. 

data, neither individuals, district offices, nor states will be identlfl 



I 



t. 







APPENDIX D 
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JOB SATISFACTION INVENTORY 



This Inventory is Another part of our study into the criteria problem in 
i .Hna «-he work of the rehabilitation counselor. We are sure that most 
::uidigrL^Lr»ti,f^io::ith . Job i. . vu.i c^t m th . ^0*.™. 
of that Job. This variable we want to include in our study. 

The following specific statement, concern your feelings, beliefs, and atti- 
tude. There «e 70 Items in this inventory. For each statement, a five-point 
scale* is provided indicating whether you' rarely, sometimes, frequently, 8« ner *y» 
or il»o’t llwa,rdo! feel, or see other, e. doing a. the statement suggests. Thus, 
for example, you would check the space "R" on the scele If you rSESll engage in 
the activity described: 

R S F G A 
I worry a lot about my job. * 



Or, see others as engaging rarely in an activity: 



The policies and problems of the 
people under whom I work are ade- 
quately explained to me. 



To eld you in enswerlng this Inventory, the terms rarely, ■°**tlmes, fre- 
quently, generally, and almost always have been defined on a percentage b 

as follows: 



•*R" 

"S" 

ttptt 

"G" 

"A" 



- Rarely: 


means 


from 0 to 


- Sometimes: 


means 


from 16 to 


- Frequently: 


means 


from 36 to 


- Generally: 


means 


from 66 to 


- Almost Always: 


means 


from 86 to 






1 






i 



.i 

■g 



% 


















one statement. 
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,, R M - Rarely (0% to 15%) 


^ 

"G" - Generally (66% to 85%) 


”S" - Sometimes (16% to 35%) 


••A" - Almost Always (86% to 100%) 


"p" - Frequently (36% to 65%) 


ZA 



Answer every question. 

1. If I had a choice, I would choose a job in my 
present line of work over one in any other line 

of work. 



R 



2. I feel that I have an adequate understanding of 
what is expected of me in my job. 

3. It i8 necessary for me to do things I dislike 
in order to get promotions. 

4. I feel that to me others could make my work 
easier if they cared to do so. 



:>. 



I worry a lot about my daily work. 



6. I feel if I could start over again, at 18, I 
would choose a different line of work. 



7. I feel that people in general respect my job. 
g # There are too many people telling me what to do. 



9. I feel that I can always trust the people under 
whom I work. 



10. My life would seem empty without my work to 
occupy me. 



11. My present job requires me to work too long hours. 

12. I am glad to get back to my job after a vacation. 



13. I feel that I am as efficient as the average per- 
son with whom I work. 



14. My work is too confining to suit me. 

15 . x feel I am paid a fair salary for the work I do. 

16. I feel that my work utilises my full capacities. 












i 
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"R" - Rarely (0% to 15%) 

"S" ~ Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"G" - Generally (66% to 85%) 

"A" - Almost Always (86% to 100%) 



Answer every question. 

R £ F G A 

0Bk» 

17. I feel that I am "in a rut" vocationally. . . ; 



18. I feel that I know where I stand with my 
present employer. 



19. I feel that my work has a bad effect on my 
health. 



20. I come home upset, angry or irritable because 
of something that happened at work. 



21. I feel competent and fully able to handle my job. 



22. I feel my work suffers because I have too much 
to do. 



23. I would decline an opportunity to change my 
present Job for one of equal pay. Security and 
status. 



24. I think It is possible to attain my vocational 
goals In that portion of life that Is still 
ahead of me. 



25. I feel that my family and friends respect my 
vocation. 

26. I feel there Is adequate transportation avail- 
able to me In going to and from work^ as well 
as in my work when called for. 

27. I think I really wanted to enter ay present Job 
when I started it. 



28. 1 regard my present position as a lifetime career. 

29. I think my present job is in the area of work 
(not necessarily the same Job) I wish to remain 
in permanently. 




, •' ^-.I'- ■. • : • '•: ; -.' : -;vW / % ;V -,y ■; *r s - ■ v: v- ;-f. v. '•••'• , v - --v* *•;/■■.■' •■■■■ : • v-;.'. ••', •' 

" ; : ff 



H R M - Rarely (0% to 15%) 

"S M - Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"G" - Generally ( 66 % to 85%) 

"A" - Almost Always ( 86 % to 100%) 



Answer every question. R S F G A 

30. I expect my job to give me more satisfaction , . . . 

the longer I have it. ■■ — " * 

31 . i feel I have had definite adequate preparation .... 

for the job I now hold. 1 ■* 

32 . I feel I have made real and lasting friends .... 

among my working associates. -* 



33. My position forces me to work with certain 
individuals whom I dislike. 

34 . I get discouraged in my present job. 

35 # x feel that my job detracts from my status in 
the community where I live. 

36. I consider my work surroundings to be as pleasant 
as they should be. 



37 # x feel I have eventual retirement security in my 
job. 

38. 1 g«t restless during working hours , and feel 

that the day is dragging endlessly. 



39 . i feel that there should be more people to help 
with the work I am doing. 

40. I like my present Job better than any other I 
have ever had. 

41. My job gives me more real personal satisfaction 
than the things I do in my spare time. 







42 . i feel my occupation forces me to live In home 
surroundings which are uncomfortable or Inade- 
quate according to my standards. 
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$0 



"R" - Rarely (0% to 15%) 

"S" - Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"G" - Generally (66% to 85%) 

"A" - Almost Always (86% to 100%) 



Answer every question. 

43. . I wonder whether the people under whom I work 
approve of my work. 

44. I think my job gets more difficult for me each 
year. 

45. My present job gets me badly flustered and jit- 
tery. 

46. The policies and problems of the people under 
whom 1 work are adequately explained to me. 

47. 1 feel that my general interests and attitudes 
are about the same as those of ay fellow workers 
who have similar jobs. 

48. The method of payment of my earnings causes me 
inconvenience • 

49. I feel at ease in the presence of the people 
under whom I work. 

50. I am so Interested in my work that 1 talk about 
it a great deal even after working hours. 

51. 1 feel I am kept from living as 1 would like be* 
cause of insufficient income. 

52. I am satisfied with the degree to which ay pre- 
sent job gives me an opportunity to express my 
own Ideas • 



R 



S 



F 



G 



A 




53. I find my work so interesting that it Is on ay 
mind a lot when I am not at work. 

54. I feel I have made a success of my job thus far 
in my career. 

55. My present job forces me to maintain too fast a 
pace. 
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"R" - Rarely (07. to 15%) 

"S" - Sometimes (16% to 35%) 
"P" - Frequently (36% to 65%) 



"G" - Generally (66% to 85%) 

"A” - Almost Always (86% to 100%) 



Answer every question. 

56. I teel that my working associates regard me as 
an equal. 

57. x feel that I must look outside my work for those 
things that make life worthwhile and interesting. 

58. My income is sufficient to meet my financial 
obligations and support my family. 

59. I feel that my associates stimulate me to do 
better work. 



60. I think my Job has "smothered" my personality. 

61. My vocational future looks promising to me. 

62. I feel that I am really interested in my present 
job. 

63. I get along well with the persons with whom I 
work on my present job. 

64. The people under whom 1 work make available the 
materials, information and assistance required 
to do my best work. 

65. I feel that the people under whom I work make 
unfair demands on my free time. 

66. I am afraid of losing my job. 

67. I feel that I will become more proficient at my 
work the longer I have it. 

68. Those with whom I work seem unreasonable in their 
dealings with me. 

69. I feel my present job helps me toward the finan- 
cial goals I have set for myself. 













"G" - Generally (66% to 85%) 



"R" - Rarely (0% to 15%) 



"S" - Sometimes (16% to 35%) 
"F" - Frequently (36% to 65%) 



"A" - Almost Always (86% to 100%) 



Answer every question. 

R S F G 

70. The people under whom I work are desirous of ~ " 

and willing to make Improvements In my working 

conditions. : ; • 



You may use the rest of this sheet for any comments you would like to make 
concerning this Inventory. Thank you for your help. 
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Correlations have bean corrected for selection on Co-Worker Ratings. 

Significant at .05 level. Significance was tested at df=54 since sampling distribution of corrected 
is unknown, but presumably more variable. 
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TABLE L 

CO RRELA TIONS BETWEEN CASE-POINTS AND NINE CRITERIA (N-41) 
CASE-POINTS 



Co -Worker Ratings 


15 


Supervisor Ratings 


28 


Present State Ratings 


32* 


Job Satisfaction 


03 


Caseload Velocitj 


05 


Sise of Caseload 


-22 


No. of 12*s 


08 


No. of 13's 


-02 


No. of 15' s 


07 



NOTE: DtelMli htvi bean oaltttd item correlations 

* Significant at .05 level. 
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